arXiv: 1507.0705lvl [cs.IT] 25Jul2015 


Weighted cumulative entropies: An extension of 

CRE and CE 


Yuri 


SuhovQ, 


Salimeh Yasaei 



Abstract. We generalize the weighted cumulative entropies (WCRE and WCE), introduced 
in [5], for a system or component lifetime. Representing properties of cumulative entropies, 
several bounds and inequalities for the WCRE is proposed. 
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1 Introduction. The weighted cumulative entropies 

An important measure of the uncertainty is entropy, commonly termed the Shannon information 
measure, unj. The current paper deals with weighted entropy; for the definition and initial results 
on weighted entropy the reader is referred to mm- 

Furthermore, the entropy of the residual lifetime Xf = [X — t\X > t] as a dynamic measure of 
uncertainty was considered in [3]. Recently, further progress as introducing cumulative entropy, 
cumulative residual entropy and weighted cumulative entropy was made in mmm- 

The purpose of this work is to obtain a number of results for weighted cumulative entropies 
in the case where the weight is a general non-negative function. 

Let X be a non-negative absolutely continuous random variable describing a component 
failure time, with the probability density function (PDF), /(x), the cumulative distribution 
function (CDF), F(x) = P(X < x ), and the survival function (SF), F(x) = P(X > x). 
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Definition 1.1 Given a function x E M i —> <f>{x) > 0, and an RV X : li, —>• R, with a PDF 
f, the weighted cumulative residual entropy (WORE) of X (or F) with weight function 
(WF) is defined by 

£%(X) = £%(F) = - f cf(x)F(\X\ > x) log F(\X\ > x)dx. (1.1) 

J R_l_ 

Note that a standard agreement 0 = 0- log 0 = 0- log oo is adopted throughout the paper. 

Given the CDF, x € M+ i->- F(x) € [0,1]. with WF <p, the weighted cumulative entropy 
(WCE) of non-negative random lifetime X is defined by 

r^X) = £™(F) = - f fi(x)F(\X\ < X ) log F(\X\ < x)dx. (1.2) 

J R+ 

Particularly when <f(x) = x the WCRE and WCE in 11.11) and HI. 2 1) can be turned out as (8) 
and (9) in In what follows, we intend to use the same abbreviation as in for the weighted 
cumulative residual and weighted cumulative entropies. 


Example 1.1 (WCRE of the uniform distribution) Consider an RV X with uniform distribution 
in the interval a < b, f(x) = —-—. Then the WCRE is the following: 


£;( f ) = - /■%( x )(i--^-)iog(i- r ^-)dx 

J a 


b — a' 


b — a' 


= (b-a) 


' b—2a 
b—a 


0((6-a)(l - y))ylogydy. 


In particular, with <j)(x ) = x, one obtains: 

1 


= 


36 (a — b) 


24a log 


——^ ) — 15a 2 6 — a 3 — 56 3 + 6 log 

a — b 


— 18 ab 2 log 


2a — b 
a — b 


+ 21 ab 2 + 6a 3 log 


a — b 


— 18a 2 61og 


2a — b 

a — b 
a 


a — b 


Example 1.2 (WCRE of the Gaussian distribution) Let g{x) be the Gaussian PDF with mean 
H and variance a 2 . Therefore, the SF is obtained as G(x) = erfc (—^). Here erfc(x) is the 
complementary error function: 

1 t 2 

erfc(x) = -j= J exp(— — )dt. 


In accordance with EH) we obtain: 


£J{G) = - [ (j)(x)erfc (-—-) log erfc{- — -)dx. 

Jo a a 
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Given an RV X with CDF F(x) and SF F(x), set, 

l - i r 1 

™F(t) = j ' (j){x)F(x)dx and mj(t) = J <t>(x)F{x)dx. 

Pictorially irip(t) represents the weighted mean inactivity time (WMIT) and then rrip(t) the 
weighted mean residual time (WMRT). 

Lemma 1.1 (Cf. Proposition 2.1 from [51].) Let X be an absolutely continuous RV. Then 

£;(F)=E(mJ(X)), (1.3) 

and 

r^F) = E(mJ{X)). (1.4) 

Proof. The proof follows directly with the same methodology in [2] but replacing <j>{x) in x. 


Definition 1.2 Given two functions x E M + i—^ F(x) E [0,1] and x E M + i->- G(x) E [0,1], the 
relative WCRE of G relative to F for given WF is defined by 

DJ{F\\G) = J f(x)F(x) log ^dx. (1.5) 

R+ 

Alternatively, Df(F\\G) can be termed as weighted Fullback Leibler divergence between SFs F, 
G with WF <j>. 

Paying homage to the Theorem 1.1 in mi. with similar methodology the following assertion 
holds true. We omit the proof. 

Theorem 1.1 Given SFs F and G in [0,1], assume that a WF x € M + i->- 4>(x) > 0 obeys 

j </)(x)[F(x) — G(x)] dx > 0. 

R+ 


Then 


Df(F\\G)>0. 


The equally occurs iff 


[2 

[ F 


l]</> = 0 for F-almost all x E M+. 


( 1 . 6 ) 
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Theorem 1.2 (Estimating the WORE via a uniform distribution, cf. Theorem 1.2 in [11].) Assume 

i 

that RV X takes at most m values and set pi = P(X = i ) and Pi = Pj ■ Suppose that for 

3 =i 


given 0 < /3 < 1 


m 

^(*) [Pi ~ Z 3 *] ^ °> 

2=1 


Then 


m mm 

~ ^)Pi log Pi - ~~ log P ^)Pi ~ ^2 Pi log *> 
2=1 2=1 2=1 

with equality iff for all i = 1... m, <fi(i) [pi — f3i\ = 0. 

Furthermore, assume for given a, /3 € M: 



(a — f3x)] dx > 0, 


The following assertion for non-negative RV X holds true: 

£™{X)<— f (f(x)F(x) log (a — f3x)dx. 
J R+ 

There the equality holds iff 4>(x)[F(x) — (a — Px )] = 0. 


Definition 1.3 Consider a random vector X = (Xi,..., X n ) : Q —>• M n with join survival 
function F(x) = P[Xj > x\,... ,X n > x n ]. The WORE and WCE for given WF are defined 
by 


(X) = - j 0(x)P[|X| > x] log P[|X| > x]dx, 

R™ 

s'l ( x ) = - J ^( x )-P[|X| < x] log P[|X| < x]dx. 


(1.7) 


There x = (x\, ..., x n ) and M" = [xi E M", Xi > 0). 

Let (X 1 ,X 2 ) E M 2 i— cj)(x \,x 2 ) be a given bivariate WF. The conditional WORE of X\ given 
X 2 is defined by 


sy{Xi\X2) 

f J.f mid/i v i ^ IV I ^ M F (l x i| > si,|X 2 | > x 2 ) J (1.8) 

= - / (/>{x 1 ,x 2 )¥( Xi > xi, X 2 > x 2 )\og - — -r-dxi dx 2 . 

.7r2_ P(X 2 \>x 2 ) 
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and the mutual WORE between non-negative random vectors X with joint SF F and marginal 
Fi, i = l,...,n by 


t 4> (X) 

= [ </>(x)-F(x)log 


%F n ) 

F(x) 


F\ fx ±) ••• F n (Xn) 


dx. 


(1.9) 


Lemma 1.2 (Bounding on conditional WORE, Cf. Lemma 1.1 in [TT].) Let X.f = (X \. X 2 ) be 
a pair of RVs with the joint SF F(x 1,2:2) and marginal SFs F\(xi), F 2 (x 2 ). Suppose that the 
WF cf obeys 



f (t>{x-i)F(x-i)[F(xi\x 2 ) - l]dx? < 0 

(1.10) 


R2_ 


Then 

^(X?)<£^(X 2 ) or, equivalently, SJfX^) > 0. 

(1.11) 

Here if> 2 = J 

0(x?)F(®i|x 2 )dxi and the equality holds true iff </>(x%)[F(xi\x 2 ) - l] = 

0 for all 

M_|_ 

x e R+. 



Furthermore, 

consider triple RVs X^ = (Xi, X 2 , X 3 ) and assume that 



[ 0(xf)F(x?)[F(x!|x 2 3 )-l]dx?. 

(1.12) 


R3_ 


Then 

^(X?)<^ 23 (X|) or, equivalently, S^X^X*) > 0. 

(1.13) 

Here f> 23 = j 

^ </>(xf \F(x-\ Xoldxi . In \1.13 1) the equality holds true iff 6(xf)[F(x 1 |xo) 

-1] =0 


R + 

for all xf € M+. 


By subscribing F in / in Theorem 1.3 in m with the same arguments, the following assertion, 
omitting the proof, is achieved. 

Theorem 1.3 (Sub-additivity of the WCRE, Cf. Theorem 1.3 in [IT].) Let X^ = (Xi,X- 2 ) be 
a pair of RVs with join SF Ffitf) and marginal survival function F\fxi), F 2 (x 2 ). Moreover 
suppose that the WF ( X\,X2 ) € K:[ i->- (f>{x 1,2:2) > 0 obeys 

J 0(xf)[F(xf) - F 1 (xi)F 2 (x 2 )\dxl > 0. (1.14) 

R2_ 
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Then 


££(X?) < q i (X 1 ) + S^(X 2 ), or, equivalently, £J(X 1 \X 2 ) < ^(Xl), 


or, equivalently, rg{X\ : X 2 ) > 0. 


XTie equality occurs iff X\, X 2 are independent modulo 0 i.e. 0(xf) 


1 - 


Fi(xi)F 2 (x 2 ) 


all xj € R+. Here 0i and 0 2 are emerging from conditional survival functions: 

i/ji= 4>(xl)F(xj\xi)dxj, i,j = 1,2, 

K+ 

For given WF £ M+ i->- 0(x 2 ) > 0, define 

0i2(x?) = J 0(xf ) F(x'31x 2 )dx's, x 2 

M+ 

ipf (*i) = J 0(x?)F , (x^|xi)dx^, xi 


(1.15) 

= 0 /or 

(1.16) 


2 € Ml 


€ 


(1.17) 


and similarly define tpV and 0y for distinct labels 1 < i,j,k < 3. Then as in KU, use 0i2 in 
(11.141) if the assumption 


J />(x?)[F(x 2 ) - Fi(xi)F 2 (x 2 )]i ? (x 3 |x 2 )dxl > 0. 


(1.18) 


holds true, Then 


syjxf x 2 )<£^{x{). (i.i9) 

Following the given assertions in HD, the analogue inequalities each requiring its own as¬ 
sumption are represented. Note that in the list of assumptions (1.15),(1.17),(1,22),(1.27) in [IT] 
swap F in /: 


by Lemma 1.1, HU: 0 < £J{X l |Xl ), assuming 1.17 (a modified form of 1.15), 
by Lemma 1.3, [11]: £fjf(Xi\ Xf) < £ff 12 {X\ \X 2 ), assuming (1.27), 
by Theorem 1.3, [11]: £^ 12 (Xi|X 2 ) < ^ 3 (Xi), assuming (1.22), 
by Lemma 1.2, [11]: £^^(Xi\X 2 ) < (Xi i3 |X 2 ), assuming (1.26), 

by Theorem 1.4, [T1J: (X 13 |X 2 ) < S^(Xi\X 2 ) + £^(X 3 \X 2 ), assuming (1.27), 

by Theorem 1.5, [U]: ^(X?) + Pfe{X 2 ) < ^(X?) + £%JX 1), assuming (1.27). 


( 1 . 20 ) 


Next we represent a number of results which are analogue assertions in m, hence proofs 
omitted. 
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Theorem 1.4 (Strong sub-additivity of the WORE). Given a triple of RVs Xf = (Xi, X 2 , X 3 ), 
assume that 


<K x i) 


E(x?) - F(x 2 ) n F{Xi\x 2 ) 

i= 1,3 


dxf > 0 . 


is fulfilled. Then 

SJ(Xl)-S^ 3 (X 2 ) < ^ 12 (X?) +^ 23 (X|). 

The equality holds iff RVs X\ and X 3 are conditionally independent X 2 . 


( 1 . 21 ) 


( 1 . 22 ) 


Theorem 1.5 (a) (Convexity of relative WORE). Given a WF x € M+ 4>(x) and A 1 A 2 € 
(0,1) with Ai + A 2 = 1, then 

X^iFtWGi) + \ 2 DJ(F 2 \\G 2 ) > DJ( A 1 F 1 + A 2 F 2 IIA 1 G 1 + A 2 G 2 ), (1.23) 

with equality iff \\\ 2 = 0 or F\ = F 2 and G± = G 2 modulo <f>. 

(b) (Data-processing inequality for relative WCRE). Let ( F,G ) be the pair of SFs and <ft a 
WF inW + . For given stochastic kernel II = (Jl(x,y),x,y € R+), set (u) = / 4>(x)U(u, x)dx. 

J R_l_ 

Then 


dkf\\G)>d^(fu\\Gu) 


(1.24) 


where (FIl)(x) = / F(u)U(u, x)du and (GIl)(a;) 

J K+ 

iff FU = F and GU = G. 


G(u)H(u,x)du. The equality occurs 


Theorem 1.6 Let Xf be a triple of RVs with joint SF _F(xf). Let xf = (x\,x 2 ,X 3 ) € !->• 

0(xf) be a WF such that X\ and X 3 are conditionally independent given X 2 modulo <f. 


(a) (Data-processing inequality for conditional WCRE). Assume inequality (11.2111 by swap¬ 
ping X 2 with X\. Then the following assertion for conditional WCREs holds true: 

^ 23 (X 3 |X 2 )<^ 13 (X 3 |X 1 ), (1.25) 

with equality iff X 2 and X 3 are independent modulo cj>. In addition assume that given WF <f and 
triple of RVs Xf obey 

[ <M x i)^( x i) 

JRf 

Then 

£^{X3\X 1 )<2£^{X3\X 2 )- (1.27) 


^2113(^21X13 - 1 


dxf < 0 


(1.26) 
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(b) (Data-processing inequality for mutual WORE). Assume inequality (11.281) : 


Then 



(1.28) 


: X 3 ) < tJJX! : X 2 ). 


(1.29) 


Here, equality in (11.291) holds iff, modulo cf>, RVs X\ and X 2 are conditionally independent given 
X,,. 


Theorem 1.7 (Concavity of the WORE). Given WF <f>, set 4>'(x) = -^-cj)(x) and <p"{x) = 
d 2 - - _ 

— 2 <^>(x). The functional F (->• SJ(1 — F) is concave function in F under following suppositions: 


(i) The WF f is non-increasing (non-decreasing) for x € [e 1] (x € [0,e 1 ]). 

(ii) For x £ [0,1] 

4>"(F-\x)) - < 0 , (1.30) 

f(F ( X J) 

here f denotes the derivative of f w.r.t. x. 


Proof. Set g(x) = xlogx, x £ [0,1], To implement the concavity property for WCRE, it 
is sufficient to prove the function (f>(F~ 1 (x)).g(x), x £ [0,1] is convex. Therefore we compute 

-^-()>(F _1 (x)). 5 (x) = (-^-(j>(F~ 1 (x)).g(x) + f(F- l (x)).^-g{x). (1.31) 

ax ax ax 


and 


XZ^F 1 (x)).g(x) 

d d 2 - d 2 - d d - 

= (*))M®) + (— ^g{x)).f{F 1 (x)) + 2—g(x).— ( J)(F \x)). 


(1.32) 


'dx 2 ' v dx 2 v ’’ ' rK \ n ' ’ { \ x 

Evidently the middle expression in RHS of above inequality is non-negative. Furthermore, note 
that 

0'(F -1 (x)) 

(1.33) 


d* 2 ^ ^ / 2 (F-!(x)) L 


^ 1 (*)) = - /( ^ 1 ( X ))_ I 

[ ct>"(F~\x)) - l^M±cf\F-\x)) 


f(F- l (x)) 

Combining (11.321) and (11.331) . under assumptions (i) and (ii) we conclude the result. □ 


















2 Additional results 


Following steps in the proof of Theorem 1 from [7], we propose the following theorem. 
Theorem 2.1 Assume for given 0 < a < oo the following integrals are finite: 


j </>(x)d: 


x < oo and 


—pa 

</>(x) x i n dx < 


oo 


( 2 . 1 ) 


(0,a)" 


rj7(o,a)™ 


2 = 1 


Then £J(X) < oo if for all i, p and some 0 < a < 1, Epf?] < oo. 

rx n 

Furthermore, set 'ip(x) = / <p(t)dt, in particular, <f(x.) = 4>(xi). Then for all a > 0 the 

Jo i =1 

assumptions \2.1\) take the form: 


pOO 

J a 


x i n d Xi < oo, VK 0 ) — V’(O) < oo. 


Proof. Following arguments given in [7], we using Holder’s inequality. Recall Step 2 in the 
Theorem 1 of [7j. For 0 < o < 1 we have 

-l 71 

P[Xi > Xi, 1 < i < n ]| log P[Xi > Xi, 1 < i < n]| < -1- TT P[Xi > x ^]» . 

l — o 


i— 1 


By multiplying both sides of above inequality in 0(x) and then integrating on R”, we obtain 

-1 ,■ n 

( x ) < YTTJf J > x *]" dx - 

R™ 

Furthermore, 


2=1 


o-l r 


C(X) < 


1 — a . 


J <K*)d: 


(0,a)" 


x T / 0(x) > Xj]“dx 

® n /(0,a) n 


( 2 . 2 ) 


2=1 


Owing to Markov inequality for p > 0, the last term of RHS in (|2.2I) is less and equal than: 


,-i 


1 — a 




L 2=1 


/(0,a) n 


</>(x) 11 Xj n dx. 


2=1 


By virtue of (12.11) this leads directly to the result. □ 


Remark. Note that in case </>(x) = f~[ the (12.21) reads 


2=1 


1 n poo 

£J(X) < - TT / > Xi]»dxi 


-i 


< 


l — o 


2=1 


n I W 0 ) ~ ^(°)) + E [^ P ] " / ” dx i \ ■ 
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Using the method in Theorem 5, [8] and arguments in Theorem 12.11 the following result is 
given. The proof of Theorem 12.21 is similar to Theorem 5 in [8] and omitted. 

Theorem 2.2 (Cf. Theorem 5 from [8].) Let the random vector X*, converges in distribution 
to the random vector X. Also suppose that f is a WF whereas \2.1\) holds true. If all X^ are 
bounded in L p then 



(2.3) 


Now we focus on the sum of independent RVs: The standard Shannon and cumulative entropies 
of a sum of independent RVs is larger than and equal of each . We show as analogues as Theorem 
2 in [7], the same result is fulfilled for WORE either. 

Theorem 2.3 Consider two non-negative and independent RVs X and Y with PDFs fx and 
fy, respectively. Then 

max <S%{X + Y). 


Here fy(x) = j fy(y)cj)(x + y)dy and swap X with Y in fx- 

Proof. We again follow the argument from [Tj. By using Jensen’s inequality, write: 

P[X + Y > w] log P[X + Y > w] 

< j fy(y)P[X >w-y] log P[X > w - y\dy. 

Multiply both sides by f(w) and then integrate with respect to iv from 0 to oo: 

/ poo 

Mv) J 4 >{w)P\X > w — y] log P[X > w - y\dwdy 

p poo ^ 

= / fy (y) / (f>(w)P[X > w-y] log P[X > w - y\dwdy 


•y u 

/ poo 

fy(y) / cj>(w + y)P[X > re] log P[X > w]dwdy 

poo r p 0 

J J <t>(w + y)f Y (y)dy 


(2.4) 


P[X > w} log P[X > w]dw = S^ Y (X). 


The first equality here is obtained because X is a non-negative RV. Consequently, for w < y, 
P[X > w — y\ = 1. □ 


In addition, the following extended assertion of Theorem 4 from [7] holds true (and is straight¬ 
forward) . 
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Theorem 2.4 For given independent RV Xi, we have 




(2.5) 


/ - n 

b>(x) II F(xj)dxi dx) ! + i • In a particular case </>(x) = fj <f>j{xj), set ipj(xj 

r x i 

/ (j)j(t)dt. ThenSJfX.) in |777|) becomes: 

Jo 


n / n 


£ ?( x )=E n 

i=l 

3 Bounds for the WCRE 


Ex,(^-(*i))-^(0) 


^ra¬ 


in this section, our goal is to establish additional bounds for the WCRE. First, let us show how 
the WCRE can be dominated by the standard entropy, as well as by the CRE; cf. [7j. 

Theorem 3.1 Let h{X) be the Shannon entropy of a non-negative RV X having PDF f and 
SFF. Then 


> a$ exp {h(X)}, 


where 


ottf, = exp |lE[log</>(X)] + J log logx|^dx|. 

Proof. The proof follows directly from log-sum inequality: 

/(*) 


(3.1) 


(3.2) 


f(x )log 


(j>(x)F(x) | log F(x) | 


> 



I J f{x)dx \ 

l J j io & 

i f f>(x)F(x )\log F(x) dx J 


= log 




Note that if J (j>{x)F{x) log F{x)dx = oo, the proof is trivial. The LHS leads: 

— h(X) — E[log<^(X)] — f logx| log a;|da;. 

Jo 


Consequently, 

log q, 

This completes the proof. □ 


(X)>h(X)+E[\og<j>(X)}+ f logx| logx|dx. 

Jo 
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Let (X,Y) be a pair of RVs with a conditional SF F(x\y). Moreover assume an additional 
WF (x, y ) € M+ i->- (p(x, y) > 0. The WCRE of RV X given Y = y with WF ip(x, y) is defined 
by 


£™(X\Y = y) = - J <p(x, y)F(x\y) log F(x\y) dx. (3.3) 

Later, owing to (13.3|) . the generalized statement of Theorem 13.11 with similar proof is driven, 
therefore we omit the proof. 


Lemma 3.1 For a non-negative RV X, let f(x\y ) be the conditional PDF X given Y = y. Set 

ot<p{y) = exp jEx|K=y[log¥>(X,y)] + J l og Rc| log dz j 


Then 


£Z{X\Y = y)> a v (y) ex.p{h(X\Y = y)}. 


where h(X\Y = y) is Shannon entropy of X given Y = y. 


(3.4) 


Definition 3.1 Given WF (x,y) G >-)• ip(x,y) > 0, set <f>(x) 
Cross WCRE is introduced by 


p(x,y)f{y\x)dy. 


The 


£%(X, Y) = £?(X) + Ex [8%(Y\X = x )]. 


(3.5) 


Lemma 3.2 Assume WF (x,y) € n- ip(x,y ) > 1 and set 


a* = exp Ex, y [log (f>(X, T)] + / log x\ log x\ dx >. 


Then 

£J(X,Y) > 2a;exp{ELL}. 

Proof. Following the assumption ip > 1, we observe 

Ex[log0(X)] > Ex,y [ log <p(X, T)]. 
Owing to the convexity of e x and Jensen inequality, we have 

®[€%(Y\X = s)] > a;exp{h(Y\X)}. 


(3.6) 


(3.7) 


(3.8) 
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where h(Y\X) is denoted for conditional standard entropy. 


£™(X,Y) = (X) + E x [8$ {Y\X = x)\ 

> a^exp {h(X)} + a* exp{h(Y\X)} 
• 'X, T " 

2 


(3.9) 


> 2a*exp{^}. 


Here is defined as (13.2 j) by replacing cj) in cj). The first inequality in (13.9(i drives from (13.ip 

_____ t s 

and (13.81) . The second inequality holds by using (|3.7I) and 2exp(—-—) < exp(f) + exp(s). □ 
Lemma 3.3 (Cf. Proposition 4 from [7].) Let X be a non-negative continuous RV. Given WF 

rx 

cf), suppose that i/j(x) = / <f(t)dt is bounded. There exist a function Y = g(X) such that: 

Jo 

(i) The WORE and the weighted entropy (WE) are related by 

J E(A') E(X) S V 

(H) Assume =0, then the WORE and the Shannon entropy (SE) are related by 

£J(X) 0 


h(Y) = 


E(^(X)) E(^(X)) 

POO 

here © = / J>{x) log J>{x) F(x)dx. 

Jo 


+ log E(V>(X)), 


Proof. The proof is straightforward by considering the CDF, F. as an RV having PDF 

^ . —- and ^ [1, ^ , respectively. Next use the definitions of SE and WE. 

E(X) E(^(X)) 

Note that simply by choosing g(x) = F~ * 1 (F(x)) we can find a g. □ 


Next we present a Lower bound for WCR, the origin of this Lemma goes back to Proposition 

1 from § 1 . 


Lemma 3.4 Let X and Y be two iid RVs. 
obtain 


rx 

Also For given WF cf) set ip(x) = / 4>(t)dt. We 

Jo 


2£$(X)>E[W(X)-iP(Y)\\. (3.10) 

In particidar, suppose that X is a non-negative RV, then 

2Sj(X)>E[MX)-E[il>(X)]\]. (3.11) 
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Proof. According to Proposition 1, [8j, similarly we derive: 


2 F(x) — 2 F 2 (x) = P[max{Jf, Y} > x\ — P[min{X, Y} > x ], (3.12) 

multiplying both sides of (13.121) in <f>(x) and then integrating from zero to infinity: 

pOO 

2 / cj)(x)F(x)( 1 — F(x))dx 

Jo 

poo poo 

= / 0(x)P[max{A', Y} > x] — / 0(x)P[min{X, Y} > x]. 

Jo Jo 

Next using integrate by part in RHS , we can write 

poo 

2/ F(s)|logF(x)|dx > E[|V>P0 - V’(^)l]- (3.13) 

J o 

The LHS can be modified becaue of x(l — x) < ®|logx|. The inequality (|3.13l) proves (13.101) . 
Moreover the assertion (13.111) follows directly from: 

E[\^X)-^(Y)\] >E[\il>(X)-EMX)]\]. □ 


Lemma 3.5 (Cf. Proposition 2 from [8j.) Let X be a non-negative RV. Then for function if 
defined as in Lemma \3Jj\ 


SJ(X)=E 


(^(O)--0(X))(l + log F(X)) 


(3.14) 


Proof. The proof is straightforward and based on the equality: 


L 


F(x)\ogF(x) = — (1 + logF(t))dF(t). 


□ 


Remark: More application of conjugate or the Fenchel Transform of the convex function 
x\ogx is exp(y — 1), that is 

exp (y — 1) = sup [xy — x log x : 0 < x < oo]. 

Consequently, for non-negative RVs X and Y : 

xy < x log x + exp (y — 1). 


If we use this inequality, emerging the definition WCRE, an upper bound for WCRE in terms 
of \if(X) — EfV’POJI is given: 


S$(X) < 2E 


MX) - ®MX)]\ log \if(x) - emx)]\ 



e 


Here if is defined as before. 
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Theorem 



3.2 (Cf. 

and i/j^ 1 


Theorem 1 from [8j.) Suppose that X is a non-negative RV. Set tf(x) = 
the inverse function of if. Then 


E[if(X) log+ if(x)] 

< S%(X)+E[1;(X)1(X > log (eEty(X)l(X > ^(l))])- 

This implies: E[if}(X) log + ip(x)] < oo if WORE is finite. 


(3.15) 


Proof. Following standard calculations, (see [8]), we can write 
E \4>{X) log + if(x)] 

p oo 

= E\-if(X)l(X > -0 _1 (1))1 — F(^~ 1 (l)) + / 4>(x)F(x)logif(x)dx. 

i) 

Moreover, for t > ^> —1 (l) one yields: 


(3.16) 


il>(t)F(X >t)< E[^(X)1(X > t)] < E[ip{X)l(X > V» _1 (l))]. 

Therefore, we obtain 

POO 

/ <f>(x)F(x) log ip(x)dx 

POO 

< £%{X) + logE[^(X)l(X > ^ _1 (1))] / <Kx)F{x) dx. 

Finally according to (13.161) . we get 

E ty(X) log+ V>(*)] < £J{X) + <;(l + log E [iJ>(X)l(X > ^(l))] j . 

Here <, = E\^f(X)l(X > 1 (1))] — F(^ _1 (l)). The inequality (13.151) holds true then. □ 


4 Maximum WCRE properties 


Theorem 4.1 Suppose x £ M + H 4>(x) > 0 is given WF. Then F m maximizes the WCRE 
£J(F), modulo cf, uniquely when the following constrains are fulfilled: 


/«,)[*(,>-**(,>] d*>0 

R + 


and 


J <f>(x)[F(x) 

R + 


F m (x)] logF m (x)dx > 0. 


(4.1) 
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Example 4.1 Consider a random vector X” = (Xi ,.... X n ) : Q —> M n with PDF f, the PDF 
F and the SF F, mean vector p = (jj .\,..., fi n ) with EX* = pLi and covariance matrix C = (Cjj) 
with Cij = E (Xj — Hi)(Xj — fij) , 1 < i, j < n. Let / No be the normal PDF with the same p 
and C and F No be the normal SF. Introduce 


a 


(x) = J exp | - i(t - /i) T C x (t - /i)}dt. 


(4.2) 


Then 


(4.3) 


p(C) := F No (x) = (2vr)- n / 2 (det C)- 1 / 2 a*(x). 

Given a WF xf = (xi, ..., x n ) € M n </>(x™) > 0, suppose that 
J <Xx) F(x) - F No (x) dx > 0 and 

R” 

log [(27r) n / 2 (det C) 1 / 2 ] J f{x) [F(x) - F No (x)] dx - J </>(x) [F(x) - F No (x)] log a*(x)dx < 0. 


Then 


£J(F) < SJ(F^) 

= i log [(27r) n (det C)] J 0(x)F No (x)dx — J 0(x)F No (x) log a*(x)dx. 


(4.5) 


with equality iff F = F No modulo (j). 


Example 4.2 Lei F Ba;p and F Exp be respectively CDF and SF on M + with mean 4-. Suppose 
the following constrains are fulfilled: 


/* (a 


F[x) - F Exp (x ) 


dx > 0 and / x 4 >{x) 


F(x) - F Exp (x) 


dx < 0, 


(4.6) 


where x E M+ i->- </>(x) > 0 is a gixen WF. T/ien 

FJ(F) < e%(F E3 *) = A I'x f(x)e- Xx dx. 

K+ 

and F Exp is a unique maximizer modulo 4>. 

The next Theorem is a direct result of Theorem 11.71 and Example 14.11 


(4.7) 


(4.4) 
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Theorem 4.2 (The weighted Ky Fan inequality using the WORE; cf. [llj . Theorem 3.2). Assume 
for given Ai, A 2 E [0,1] with X\ + A 2 = 1 and positive definite matrices Ci, C 2 and C = Ci + C 2 
the assumption in Theorem (i) and (ii) hold true. Furthermore 


J </>(x) [AiTq 1 (x) + A 2 Ec 2 (x) - Fq(x)] dx > 0 and 

R™ 

log [(27r)"/ 2 (det C) 1 ^] j 0( x ) [^(x) + A 2 E° 2 (x) - F°(x)]dx 

- J 0(x)[AiF’c 1 (x) + A 2 F , c 2 ( x ) - ^c( x )] logac( x ) dx < °- 


(4.8) 


are fulfilled. Then 


p (AiCi + A 2 C 2 ) - A lP (Ci) - A 2 p(C 2 ) > 0. 


with equality iff A 1 A 2 = 0 or C\ = C 2 . 


(4.9) 


Lemma 4.1 Let X” = {X \,..., X n ) 6e a random vector, with components Xj : Q —> T), 1 < 
i < n, and the joint SF F. Introduce the random vector = (x) -1 , x" +1 ), Ffixf) the marginal 
SF for RV Xi : 

Fix'!) 

Ffixf) = _lim F(x) and FXx^Xi) = —-. 

xi—>00 TfiXi) 


For given a WF cf>, suppose that 


j <K x ) - n^( x *) 

TD>n 1 


dx > 0. 


(4.10) 


Then 


where 


^(x)<£^pq 


2=1 


iffixi) = J 0(xi )T|j( X ”|xi)dx^ 1 


ax i+l- 


(4.11) 


(4.12) 


77ie equality in \4-ll\) holds true holds iff, modulo <f>, components X\, \dots,X n are independent. 
In the following theorem a straightforward application of Lemma ll.ll is given. 
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Theorem 4.3 (The weighted Hadamard inequality using the WORE; cf. [11], Theorem 3.3). Let 
C = (Cij) be a positive definite nx n matrix and F^° the normal SF with the zero mean vector 


and the covariance 

matrix C. For 

given WF x” = (aq ,. 

. ,X n ) € M n H- 

introduce a*(x) 

by S4-*ty and 






OO 

a*(x ) = / e~ l / 2Cii d t and a= f 

</>(x)F No (x) dx. 

(4.13) 


X 

Rif. 



Suppose that 






[ <K x ) 

n 

>°(x)-n^w 

dx > 0. 

(4.14) 


J 

R™ 

i=i 




Then 


a 

2 


log 


]JC, 


'det C 


/« 


+ / ^(x)F No (x) log 


a 


! (x)/n^ 


dx > 0, 


(4.15) 


with equality iff C is diagonal. 


Next, we provide a characterization of the Weibull distribution using the maximum WORE. 


r 

Theorem 4.4 (Cf. Theorem 2 from [8].) Suppose (x) = / t p fi{f)dt is a non-negative WF 

Jo 

nx 

such that fi>(x) = / fi{t)dt and x € M + i->- € [0,1]. Among all non-negative RVs with 

Jo 

given E[-0(X)] and E[fi*(X)] the Weibull distribution W with SF F^^(t) = exp(— \ q t q ), has the 
maximal WORE. 

Here the parameters q = p and 

„=( _a_ Y( mxFm v (4i 6) 

where c p = T(1 + |). 

Proof. The subsequence argument works by using Log-sum inequality once more. Accord¬ 
ing to (21) in [8] but replacing G(x) = Ewib(^) = exp(— p p x p ) in ill.51) . we get 






log 


n^x)] - m 


h 1 Cp 


+ 


poo 

p p / fi(t)t p F(t)dt. 

Jo 


Now choose p 1 c p = E[^(A)] — ^(0): 


, <j(nr r (x)\-rM 


18 














Finally let q = p and A as in (14.161) . therefore we have 


£%(X) < A^Ewib ir q (x)} - r q ( 0)) = e$(Wib). 

This completes the proof. □ 


rx 

Theorem 4.5 Suppose that functions if and (j> are as in Theorem 3.2: if(x) = / <f>(t)dt), 

Jo 

and 0 < <j>(x) < 1. Let X be a given non-negative RV. In addition assume Z := X(X) is an 
exponentially distributed RV with mean A -1 = E[-0(X)] — ^(0). If the constraints 

dx>0. (4.17) 

R + 


holds true, then 

£J(X) < £J(X(X)) = A jx 0(x)F Ex P( x)dx. 

R + 


(4-18) 


Proof. Using log-sum inequality we obtain 

J <f>(x)F(x) log [<f(x)F(x)e x x )da 


> ( / <j>{x)F{x)dx ] log ( A / 4>(x)F(x)dx 


We also can write 


E [ip(X)\ - ^(0) = / <j>(x)F(x)dx, 


therefore the expression (14.191) becomes 

— £^(F) + J F{x)(j>{x) log 4 >(x)dx + A J x 4>(x)F(x)dx 

R+ R + 

> (E[^(X)] - V’(O)) | log A + log (E[^(X)] - ^(0)) 

Equivalently 

—£jf(F)> — f F(x)cf>(x) log (f(x)dx — A f x cf>(x)F(x)dx 

R + R+ 

+ (E[^(X)] - V>(0)) | log A + log (E[V>P0] - V>(0)) \ • 

Now, set 


w = E[a/>(X)] — 0) ( / x <p(x)F(x)dx) , A* = 


-l 


VO 


E[V>(X)] - ip(0) 


(4.19) 


(4.20) 
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It is admissible (14.201) is fulfilled for all positive A, so is also valid for maximum value of A = A*. 
This represents the formula: 

> — f F{x)(f>{x) log 4>(x)dx — [E[^(Al)] — -0(0)] + [E[0(X)] — 0(0)] logzu 

K+ 

> - [E[0(A)] - 0(0)] + [E[0(A)] - 0(0)] log ro 

> - wpo] - ^(o)] + - V'(o)] 11 -11 

= -[E[0(A)] -0(O)]ro _1 . 

The second inequality holds true owing to 0 € [0,1] and the last inequality is satisfied by using 
logx > 1 — x € M+. 

Recalling assumption (14.171) . leads to (14.181) . □ 
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